Today is Easter Sunday, Prime minister of United Kindom Boris Johnson recovered from COVID-19. Canada has been suffereing COVID-19 for a month already. I’m re-writing this blog FIRSTLY written in September 2019, and UPDATED ONCE in December 2019.
Merry Christmas and happy new year everybody. I’ve been back to Vancouver for several days. These days, I’m updating this blog FIRSTLY written in September 2019. 2020 is coming, and we’re getting 1 year older. A kind of sad hum?
😝
Okay… No matter what, let’s enjoy the song first: WE ARE YOUNG. Today, I joined Free Software Foundation and start my journey of supporting Open Source Software BY CASH. For me, it’s not about poverity or richness. It’s ALL about FAITH.
To write something about Raspberry Pi is to say GOOD BYE to my Raspberry Pi 3B, and WELCOME Raspberry Pi 4 at the same time. Our target today is to build an AI edge computing end as the following Youtube video:
1. About Raspberry Pi 4
1.1 Raspberry Pi 4 vs. Raspberry Pi 3B+
Before we start, let’s carry out a simple comparison between Raspberry Pi 4 and Raspberry Pi 3B+.
1.2 Raspbian Installation
1 | ➜ raspbian sudo dd bs=4M if=2020-02-13-raspbian-buster.img of=/dev/mmcblk0 conv=fsync |
1.3 BCM2711 is detected as BCM2835
1 | ➜ ~ cat /proc/cpuinfo |
This issue seems to be a well-known bug. Raspberry Pi 4’s specification can be retrieved from The MagPi Magazine. More details about the development history of Raspberry Pi can be found on Wikipedia.
2. Movidius Neural Compute Stick on Raspberry Pi 4
Then, we just follow the following 2 blogs Run NCS Applications on Raspberry Pi and Adding AI to the Raspberry Pi with the Movidius Neural Compute Stick to test out Intel Movidius Neural Compute Stick 2:
| Intel Movidius Neural Compute Stick 2 | Intel Movidius Neural Compute Stick 1 |
|---|---|
 |
 |
Intel Movidius Neural Compute Stick 1 is NOT listed on Intel’s official website any more. But github support for Intel Movidius Neural Compute Stick 1 can be found at https://github.com/movidius/ncsdk.
2.1 NCSDK Installation
We FIRST need to have ncsdk installed. Yup, here, as described in Run NCS Applications on Raspberry Pi, we carry out the installation directly under folder ...../ncsdk/api/src.
1 | ➜ src make |
2.2 Test NCSDK Example Apps
2.2.1 For Movidius NCS 1
1 | ➜ hello_ncs_py lsusb |
2.2.2 For Movidius NCS 2
1 | ➜ hello_ncs_py lsusb |
The above bug has ALREADY been expalined in main online resource:
- ncDeviceOpen:527 ncDeviceOpen() XLinkBootRemote returned error 3
- Make the examples Error! on NCS2 - [Error 7] Toolkit Error: USB Failure. Code: Error opening device
- etc.
All these hint that OpenVINO should be utilized instead of NCSDK2.
2.3 mvnc Python Package
1 | ➜ ~ python |
3. Transitioning from Intel Movidius Neural Compute SDK to Intel OpenVINO
By following Intel’s official documentation Transitioning from Intel® Movidius™ Neural Compute SDK to Intel® Distribution of OpenVINO™ toolkit, we are transitioning to OpenVINO, which supports both Intel NCS 2 and Intel NCS 1.
3.1 Install OpenVINO for Raspbian
For the installation details of OpenVINO, please refer to the following 2 documentations:
We now extract the MOST up-to-date l_openvino_toolkit_runtime_raspbian_p_2020.1.023.tgz under folder /opt/intel/openvino. Let’s take a brief look at this folder:
1 | ➜ openvino pwd |
Clearly, by comparing with OpenVINO™ Toolkit - Deep Learning Deployment Toolkit repository, we know that the open source version of deployment_tools contains some more content than the trimmed version for Raspbian. We’ll use model-optimizer for sure. Therefore, we checked out dldt, and put it under folder /opt/intel.
1 | ➜ intel pwd |
3.2 Build OpenVINO Samples
Before start building OpenVINO samples, please have OpenCV built from source and installed.
Note: Be sure to enable -DCMAKE_CXX_FLAGS=’-march=armv7-a’ while building dldt/samples, which is exactly the same as the source under l_openvino_toolkit_runtime_raspbian_p_2020.1.023/inference_engine/samples/cpp. In fact, to build l_openvino_toolkit_runtime_raspbian_p_2020.1.023/inference_engine/samples/c, -DCMAKE_CXX_FLAGS=’-march=armv7-a’ also needs to be enabled.
After having successfully built C/C++ samples, let’s enter folder /opt/intel/openvino/inference_engine/samples.
3.3 Device Query
3.2.1 C
There is NO such an exe file hello_query_device_c.
3.2.2 C++
For NCS 2
1 | ➜ samples ./cpp/build/armv7l/Release/hello_query_device |
For NCS 1
1 | ➜ samples ./cpp/build/armv7l/Release/hello_query_device |
3.2.3 Python
For NCS 2
1 | ➜ samples python ./python/hello_query_device/hello_query_device.py |
For NCS 1
1 | ➜ samples python ./python/hello_query_device/hello_query_device.py |
3.3 Object Detection
Please refer to Device-specific Plugin Libraries for ALL possible device types.
We then download two model files as given in the blog Install OpenVINO™ toolkit for Raspbian* OS.
Please note that:
- face-detection-adas-0001.xml and face-detection-adas-0001.bin are not working properly. Please refer to OpenVINO Issue 848966.
- For now, please use the following model files from year 2019: face-detection-adas-0001.xml and face-detection-adas-0001.bin.
Afterwards, start running Object Detection for 2 images: me.jpg and parents.jpg:
3.3.1 C
For NCS 2
1 | ➜ samples ./c/build/armv7l/Release/object_detection_sample_ssd_c -m face-detection-adas-0001.xml -d MYRIAD -i parents.jpg |
1 | ➜ samples ./c/build/armv7l/Release/object_detection_sample_ssd_c -m face-detection-adas-0001.xml -d MYRIAD -i me.jpg |
For NCS 1
1 | ➜ samples pwd |
1 | ➜ samples ./c/build/armv7l/Release/object_detection_sample_ssd_c -m face-detection-adas-0001.xml -d MYRIAD -i me.jpg |
3.3.2 C++
For NCS 2
1 | ➜ samples ./cpp/build/armv7l/Release/object_detection_sample_ssd -m face-detection-adas-0001.xml -d MYRIAD -i me.jpg |
1 | ➜ samples ./cpp/build/armv7l/Release/object_detection_sample_ssd -m face-detection-adas-0001.xml -d MYRIAD -i parents.jpg |
For NCS 1
1 | ➜ samples ./cpp/build/armv7l/Release/object_detection_sample_ssd -m face-detection-adas-0001.xml -d MYRIAD -i me.jpg |
1 | ➜ samples ./cpp/build/armv7l/Release/object_detection_sample_ssd -m face-detection-adas-0001.xml -d MYRIAD -i parents.jpg |
3.3.3 Python
For NCS 2
1 | ➜ samples python ./python/object_detection_sample_ssd/object_detection_sample_ssd.py -m face-detection-adas-0001.xml -d MYRIAD -i me.jpg |
1 | ➜ samples python ./python/object_detection_sample_ssd/object_detection_sample_ssd.py -m face-detection-adas-0001.xml -d MYRIAD -i parents.jpg |
For NCS 1
1 | ➜ samples python ./python/object_detection_sample_ssd/object_detection_sample_ssd.py -m face-detection-adas-0001.xml -d MYRIAD -i me.jpg |
1 | ➜ samples python ./python/object_detection_sample_ssd/object_detection_sample_ssd.py -m face-detection-adas-0001.xml -d MYRIAD -i parents.jpg |
3.3.4 Results
| Parents At First Starbucks | Parents Face Detection By Object Detection SSD NCS 1 | Parents Face Detection By Object Detection SSD NCS 2 |
|---|---|---|
 |
 |
 |
| Me | Me Face Detection By Object Detection SSD NCS 1 | Me Face Detection By OpenCV DNN NCS 2 |
 |
 |
 |
3.4 Model Optimization
In this section, we are going to test out another OpenVINO example: Image Classification C++ Sample Async. After reading this blog, it wouldn’t be hard for us to notice the MOST important thing we’re missing here is the model file alexnet_fp32.xml. Let’s just keep it in mind for now.
And, let’s review a bit about our previous example: Object Detection – we downloaded face-detection-adas-0001 model from online and use it directly. So, questions:
- Are we able to download alexnet_fp32.xml from online this time again?
- Where can we download a whole bunch of open source models?
3.4.1 Open Model Zoo
It wouldn’t be hard for us to google out OpenVINO™ Toolkit - Open Model Zoo repository, under which model face-detection-adas-0001 is just sitting there. However, face-detection-adas-0001.xml and face-detection-adas-0001.bin are missing.
Let’s checkout open_model_zoo and put it under folder /opt/intel.
1 | ➜ intel pwd |
Then, let’s enter folder face-detection-adas-0001 under open_model_zoo and take a look:
1 | ➜ face-detection-adas-0001 git:(master) pwd |
It seems that each folder under intel and public contains the detailed info of each model. For instance, file intel/face-detection-adas-0001/model.yml contains all the info about model face-detection-adas-0001. However, what we really need are a .xml file and a .bin file. In the following, we are going to generate such 2 files, which are optimized specifically for movidius by following Intel OpenVINO toolkit issue 798441.
3.4.2 Download Caffe Model Files
1 | ➜ downloader git:(master) pwd |
Clearly, three files including a large model file alexnet.caffemodel has been downloaded.
3.4.3 Model Optimization
Now, are are going to optimize the downloaded caffe model and make it feedable to OpenVINO.
1 | ➜ model-optimizer git:(2020) ✗ pwd |
And, let’s take a look at what’s generated under folder /opt/intel/dldt/model-optimizer.
1 | ➜ model-optimizer git:(2020) ✗ pwd |
Note: You may meet the following ERRORs during model optimization.
1 | [ ERROR ] |
Clearly, for networkx, the ERROR message is a kind of ridiculous.
Anyway, if you meet the above 2 errors, please DOWNGRADE your packages as follows:
1 | ➜ ~ pip install protobuf==3.6.1 --user |
3.5 Image Classification
3.5.1 C
For NCS 2
1 | ➜ samples ./c/build/armv7l/Release/hello_classification_c /opt/intel/dldt/model-optimizer/alexnet.xml me.jpg HETERO:MYRIAD |
1 | ➜ samples ./c/build/armv7l/Release/hello_classification_c /opt/intel/dldt/model-optimizer/alexnet.xml parents.jpg HETERO:MYRIAD |
For NCS 1
1 | ➜ samples ./c/build/armv7l/Release/hello_classification_c /opt/intel/dldt/model-optimizer/alexnet.xml me.jpg HETERO:MYRIAD |
1 | ➜ samples ./c/build/armv7l/Release/hello_classification_c /opt/intel/dldt/model-optimizer/alexnet.xml parents.jpg HETERO:MYRIAD |
3.5.2 C++
For NCS 2
1 | ➜ samples ./cpp/build/armv7l/Release/hello_classification /opt/intel/dldt/model-optimizer/alexnet.xml me.jpg HETERO:MYRIAD |
1 | ➜ samples ./cpp/build/armv7l/Release/hello_classification /opt/intel/dldt/model-optimizer/alexnet.xml parents.jpg HETERO:MYRIAD |
For NCS 1
1 | ➜ samples ./cpp/build/armv7l/Release/hello_classification /opt/intel/dldt/model-optimizer/alexnet.xml me.jpg HETERO:MYRIAD |
1 | ➜ samples ./cpp/build/armv7l/Release/hello_classification /opt/intel/dldt/model-optimizer/alexnet.xml parents.jpg HETERO:MYRIAD |
3.5.3 Python
For NCS 2
1 | ➜ samples python ./python/classification_sample/classification_sample.py -m /opt/intel/dldt/model-optimizer/alexnet.xml -i me.jpg -d HETERO:MYRIAD -nt 10 |
1 | ➜ samples python ./python/classification_sample/classification_sample.py -m /opt/intel/dldt/model-optimizer/alexnet.xml -i parents.jpg -d HETERO:MYRIAD -nt 10 |
For NCS 1
1 | ➜ samples python ./python/classification_sample/classification_sample.py -m /opt/intel/dldt/model-optimizer/alexnet.xml -i me.jpg -d HETERO:MYRIAD -nt 10 |
1 | ➜ samples python ./python/classification_sample/classification_sample.py -m /opt/intel/dldt/model-optimizer/alexnet.xml -i parents.jpg -d HETERO:MYRIAD -nt 10 |
3.5.4 Results
Classification Result for me.jpg
| Result from NCS 2 | Result from NCS 1 |
|---|---|
| 838: ‘sunscreen, sunblock, sun blocker’, | 978: ‘seashore, coast, seacoast, sea-coast’, |
| 978: ‘seashore, coast, seacoast, sea-coast’, | 977: ‘sandbar, sand bar’, |
| 977: ‘sandbar, sand bar’, | 838: ‘sunscreen, sunblock, sun blocker’, |
| 975: ‘lakeside, lakeshore’, | 975: ‘lakeside, lakeshore’, |
| 903: ‘wig’, | 903: ‘wig’, |
| 638: ‘maillot’, | 638: ‘maillot’, |
| 976: ‘promontory, headland, head, foreland’, | 433: ‘bathing cap, swimming cap’, |
| 433: ‘bathing cap, swimming cap’, | 976: ‘promontory, headland, head, foreland’, |
| 112: ‘conch’, | 112: ‘conch’, |
| 639: ‘maillot, tank suit’, | 639: ‘maillot, tank suit’, |
Classification Result for parents.jpg
| Result from NCS 2 | Result from NCS 1 |
|---|---|
| 707: ‘pay-phone, pay-station’, | 707: ‘pay-phone, pay-station’, |
| 577: ‘gong, tam-tam’, | 577: ‘gong, tam-tam’, |
| 704: ‘parking meter’, | 813: ‘spatula’, |
| 955: ‘jackfruit, jak, jack’, | 704: ‘parking meter’, |
| 813: ‘spatula’, | 910: ‘wooden spoon’, |
| 910: ‘wooden spoon’, | 515: ‘cowboy hat, ten-gallon hat’, |
| 515: ‘cowboy hat, ten-gallon hat’, | 955: ‘jackfruit, jak, jack’, |
| 605: ‘iPod’, | 523: ‘crutch’, |
| 918: ‘crossword puzzle, crossword’, | 808: ‘sombrero’, |
| 523: ‘crutch’, | 918: ‘crossword puzzle, crossword’, |
classid table:
4. OpenCV DNN with OpenVINO’s Inference Engine
For the example openvino_fd_myriad.py given on Install OpenVINO™ toolkit for Raspbian* OS, the TOUGH thing is how to build OpenCV with OpenVINO’s Inference Engine. NEVER forget to export export InferenceEngine_DIR="/opt/intel/openvino/inference_engine/share" and enable WITH_INF_ENGINE ON to have OpenCV rebuilt. Before moving forward, my modified version of openvino_fd_myriad.py is provided as follows:
1 | import sys, argparse |
4.1 Intel64
On my laptop, of course, we are building OpenCV for architecture Intel64, with NVidia GPU + CUDA support, for DNN in OpenCV requires either CUDA or OpenCL.
My test result of openvino_fd_myriad.py shows the performance of adopted model face-detection-adas-0001 is NOT as good as expected.
For NCS 2
1 | ➜ OpenVINO_Examples python ./openvino_fd_myriad.py -x /opt/intel/openvino/inference_engine/samples/cpp/build_18.04/intel64/Release/face-detection-adas-0001.xml -b /opt/intel/openvino/inference_engine/samples/cpp/build_18.04/intel64/Release/face-detection-adas-0001.bin -i ./parents.jpg |
For NCS 1
1 | ➜ OpenVINO_Examples python ./openvino_fd_myriad.py -x /opt/intel/openvino/inference_engine/samples/cpp/build_18.04/intel64/Release/face-detection-adas-0001.xml -b /opt/intel/openvino/inference_engine/samples/cpp/build_18.04/intel64/Release/face-detection-adas-0001.bin -i ./parents.jpg |
Results
| Parents At First Starbucks | Parents Face Detection By OpenCV DNN NCS 1 | Parents Face Detection By OpenCV DNN NCS 2 |
|---|---|---|
|
 |
 |
| Me | Me Face Detection By OpenCV DNN NCS 1 | Me Face Detection By OpenCV DNN NCS 2 |
|
 |
 |
4.2 armv7l
For NCS 2
1 | ➜ Programs python ./openvino_fd_myriad.py -x /opt/intel/openvino/inference_engine/samples/face-detection-adas-0001.xml -b /opt/intel/openvino/inference_engine/samples/face-detection-adas-0001.bin -i ./parents.jpg |
For NCS 1
1 | ➜ Programs python ./openvino_fd_myriad.py -x /opt/intel/openvino/inference_engine/samples/face-detection-adas-0001.xml -b /opt/intel/openvino/inference_engine/samples/face-detection-adas-0001.bin -i ./parents.jpg |
Results
| Parents At First Starbucks | Parents Face Detection By OpenCV DNN NCS 1 | Parents Face Detection By OpenCV DNN NCS 2 |
|---|---|---|
|
 |
 |
| Me | Me Face Detection By OpenCV DNN NCS 1 | Me Face Detection By OpenCV DNN NCS 2 |
|
 |
 |
5. My Built Raspbian ISO With OpenCV4 + OpenVINO
Finally, you are welcome to try my built image rpi4-raspbian20200213-opencv4.3-openvino2020.1.023-ncsdk2.10.01.01.img, which is about 9.5G and composed of:
Everything has ALREADY been updated and built successfully, ONLY EXCEPT object_detection_sample_ssd. In addition, rpi4-raspbian20200213-opencv4.3-openvino2020.1.023-ncsdk2.10.01.01.img ALSO works properly on my Raspberry Pi 3 Model B Version 1.2 manufactured in 2015.
